Attention Correctness in Neural Image Captioning

Attention mechanisms have recently been introduced in deep learning for various tasks in natural language processing and computer vision. But despite their popularity, the "correctness" of the implicitly-learned attention maps has only been assessed qualitatively by visualization of several examples. In this paper we focus on evaluating and improving the correctness of attention in neural image captioning models. Specifically, we propose a quantitative evaluation metric for the consistency between the generated attention maps and human annotations, using recently released datasets with alignment between regions in images and entities in captions. We then propose novel models with different levels of explicit supervision for learning attention maps during training. The supervision can be strong when alignment between regions and caption entities are available, or weak when only object segments and categories are provided. We show on the popular Flickr30k and COCO datasets that introducing supervision of attention maps during training solidly improves both attention correctness and caption quality, showing the promise of making machine perception more human-like.Comment: To appear in AAAI-17. See http://www.cs.jhu.edu/~cxliu/ for supplementary materia

Research and simulation of fast, strong exothermic reaction in gas-solid fluidized bed about temperature distribution and hot spot problem

Gas-solid fluidized bed is widely used in petro-chemical and coal-chemical industry and other fields because of its superior heat transfer and mass transfer performances. In consideration of these performances, it is generally believed that there is a uniform temperature distribution and no hot spot in gas-solid fluidized bed compared with fixed bed. But in real industrial processes of fast, strong exothermic reactions, there are great axial and radial temperature differences and even hot spots in gas-solid fluidized bed. In this study, two-dimensional diffusion model based upon the momentum and energy conservation equations was successfully used to compute the temperature distribution of aniline reaction in fluidized bed. The result is in good agreement with real industrial measurement. In addition, this study discussed the influence of velocity and fluidized bed diameter on the temperature distribution. The result showed that in contrast to the fixed bed, increasing gas velocity during turbulent region in fluidized bed would help eliminate hot spot and reduce temperature difference. Finally, based on the comprehensive consideration of velocity and diameter, this study showed a stability region for scaling up of gas-solid fluidized bed with fast, strong exothermic reactions which helps to guide the practical operation. Please click Additional Files below to see the full abstract

Stability analysis of gas solids separation in scaling-up fluidized bed reactors

In large industrial fluidized bed reactors with high gas solids flow rates, small cyclones working in parallel are often preferred to achieve higher efficiency in the case of uniform distribution of gas-solid two-phase flow across each inlet. However, there is mounting evidence1-5 that gas-solid suspensions pass through identical paths in parallel can be significantly non-uniform, resulting in a dramatically drop in overall efficiency. In this study we used the direct Liapunov method by considering the interaction between gas and solids to detect the instability of uniformity. Owing to the special symmetry in this system, the criterion can be simplified into identifying the concavity (concave or convex) of pressure drop across a single cyclone with respect to operational parameter CT. Then, based on the stability analysis of uniformity, a novel design principle is provided to prevent non-uniform distribution at high dust loading. The effect of geometrical factor, i.e. dimensionless vortex finder diameter dr, on the stability of uniformity has been further investigated. The phase diagram of stability is calculated to give a clue of designing robust parallel cyclones system. Please click Additional Files below to see the full abstract

Progressive Neural Architecture Search

We propose a new method for learning the structure of convolutional neural networks (CNNs) that is more efficient than recent state-of-the-art methods based on reinforcement learning and evolutionary algorithms. Our approach uses a sequential model-based optimization (SMBO) strategy, in which we search for structures in order of increasing complexity, while simultaneously learning a surrogate model to guide the search through structure space. Direct comparison under the same search space shows that our method is up to 5 times more efficient than the RL method of Zoph et al. (2018) in terms of number of models evaluated, and 8 times faster in terms of total compute. The structures we discover in this way achieve state of the art classification accuracies on CIFAR-10 and ImageNet.Comment: To appear in ECCV 2018 as oral. The code and checkpoint for PNASNet-5 trained on ImageNet (both Mobile and Large) can now be downloaded from https://github.com/tensorflow/models/tree/master/research/slim#Pretrained. Also see https://github.com/chenxi116/PNASNet.TF for refactored and simplified TensorFlow code; see https://github.com/chenxi116/PNASNet.pytorch for exact conversion to PyTorc

Clinical markers predict the efficacy of several immune checkpoint inhibitors in patients with non-small cell lung cancer in China

ObjectivesImmune checkpoint inhibitors (ICIs) are one of the most significant oncological treatment modalities as a result of the rapid advancement of immunotherapy. Programmed Cell Death-Ligand 1 (PD-L1) and tumor mutational burden (TMB) have emerged as key markers for predicting the efficacy and prognosis of ICIs in non-small cell lung cancer (NSCLC), and the predictive role of tumor-infiltrating lymphocytes (TILs) has also received significant attention. However, the prognosis of some individuals cannot be determined by these indicators; for instance, some patients with low PD-L1 expression also benefit from longer survival. Therefore, the purpose of this research was to investigate the connection between new haematological and pathological markers and clinical outcomes in NSCLC patients receiving ICIs.MethodsSeventy-six patients with stage III-IV NSCLC treated with ICIs were included in this study. We used the Mann-Whitney test, COX regression and Kaplan-Meier analysis to retrospectively analyze peripheral blood indicators and survival prognostic data of 76 patients in order to investigate the relationship between baseline neutrophil-to-lymphocyte ratio (NLR) and the efficacy of ICIs. To investigate the correlation between CXCL13, CXCR5, CD8 and the efficacy of ICIs, we assessed the expression levels of aforementioned indicators in biopsied tissues of 10 non-small cell lung tumors by immunohistochemistry (IHC) and immunofluorescence (IF) and performed statistical analysis.ResultsDisease control rate (DCR) was higher in patients with baseline NLR <3.4 (p=0.016) and neutrophil percentage <71% (P=0.015). Baseline NLR (HR=2.364, P=0.003) and neutrophil percentage (HR=2.824, P=0.013) had the greatest influence on patients’ survival prognosis, with baseline NLR exhibiting a stronger predictive value (AUC=0.717), according to univariate and multifactorial COX regression analyses of progression-free survival (PFS) and overall survival (OS). In NSCLC tissues, higher expression of CXCL13 was associated with better clinical outcomes (P=0.032) and higher expression of CD8 was associated with prolonged survival (P=0.022).ConclusionLow baseline NLR in peripheral blood and high expression of CD8 in tissues are associated with longer PFS and may have a potential predictive value for patients with stage III-IV NSCLC using ICIs

Niacin downregulates chemokine (c-c motif) ligand 2 (CCL2) expression and inhibits fat synthesis in rat liver cells

Purpose: To elucidate the role of chemokine (c-c motif) ligand 2 (CCL2) in fat metabolism in hepatocytes. Methods: Following partial hepatectomy, regenerated rat liver cells were isolated and cultured for 24 h were transfected with recombinant plasmid pEGFP-N1-CCL2 using liposomes. Niacin was added to the culture medium to inhibit fat synthesis. CCL2 expression was measured using western blot, while the expression of acly-coa synthetase long chain family 4 (ACSL4) and apolipoprotein E (ApoE) were assessed using real-time PCR. Results: At 12 h after transfection, GFP-positive rates in the pEGFP-N1 and pEGFP-N1-CCL2 transfection groups were 42.4 ± 5.6 % and 45.1 ± 3.5 %, respectively. Expression levels of CCL2 increased over time in pEGFP-N1 transfection group, pEGFP-N1-ccl2 transfection group, and niacin and pEGFP-N1-ccl2 transfection co-treatment group; however, CCL2 expression levels in the niacin and pEGFP-N1-ccl2 transfection co-treatment groups were similar to that of pEGFP-N1 transfection group, which were significantly lower than those of the pEGFP-N1-ccl2 transfection group. Expressionlevel trends of fat-related genes ACSL4 and ApoE were similar to that of CCL2. Conclusion: Niacin downregulates the expression of CCL2, thereby inhibiting lipid synthesis in liver cells. Keywords: Chemokine 2, Niacin, Hepatectomy, Lipid synthesis, Transfectio

A novel WebVR-Based lightweight framework for virtual visualization of blood vasculum

With the arrival of the Web 2.0 era and the rapid development of virtual reality (VR) technology in recent years, WebVR technology has emerged as the combination of Web 2.0 and VR. Moreover, the concept of “WebVR + medical science”is also proposed to advance medical applications. However, due to the limited storage space and low computing capability of Web browsers, it is difficult to achieve real-time rendering of large-scale medical vascular models on the Web, let alone large-scale vascular animation simulations. The framework proposed in this paper can achieve virtual display of the medical blood vasculum, including lightweight processing of the vasculum and virtual realization of blood flow. This innovative framework presents a simulation algorithm for the virtual blood path based on the Catmull-Rom spline. The mechanisms of progressive compression and online recovery of the lightweight vascular structure are further proposed. The experimental results show that our framework has a shorter browser-side response time than existing methods and achieves efficient real-time simulation

A new framework for the integrative analytics of intravascular ultrasound and optical coherence tomography images

Abstract:The integrative analysis of multimodal medical images plays an important role in the diagnosis of coronary artery disease by providing additional comprehensive information that cannot be found in an individual source image. Intravascular ultrasound (IVUS) and optical coherence tomography (IV-OCT) are two imaging modalities that have been widely used in the medical practice for the assessment of arterial health and the detection of vascular lumen lesions. IV-OCT has a high resolution and poor penetration, while IVUS has a low resolution and high detection depth. This paper proposes a new approach for the fusion of intravascular ultrasound and optical coherence tomography pullbacks to significantly improve the use of those two types of medical images. It also presents a new two-phase multimodal fusion framework using a coarse-to-fine registration and a wavelet fusion method. In the coarse-registration process, we define a set of new feature points to match the IVUS image and IV-OCT image. Then, the improved quality image is obtained based on the integration of the mutual information of two types of images. Finally, the matched registered images are fused with an approach based on the new proposed wavelet algorithm. The experimental results demonstrate the performance of the proposed new approach for significantly enhancing both the precision and computational stability. The proposed approach is shown to be promising for providing additional information to enhance the diagnosis and enable a deeper understanding of atherosclerosis

AutoPoster: A Highly Automatic and Content-aware Design System for Advertising Poster Generation

Advertising posters, a form of information presentation, combine visual and linguistic modalities. Creating a poster involves multiple steps and necessitates design experience and creativity. This paper introduces AutoPoster, a highly automatic and content-aware system for generating advertising posters. With only product images and titles as inputs, AutoPoster can automatically produce posters of varying sizes through four key stages: image cleaning and retargeting, layout generation, tagline generation, and style attribute prediction. To ensure visual harmony of posters, two content-aware models are incorporated for layout and tagline generation. Moreover, we propose a novel multi-task Style Attribute Predictor (SAP) to jointly predict visual style attributes. Meanwhile, to our knowledge, we propose the first poster generation dataset that includes visual attribute annotations for over 76k posters. Qualitative and quantitative outcomes from user studies and experiments substantiate the efficacy of our system and the aesthetic superiority of the generated posters compared to other poster generation methods.Comment: Accepted for ACM MM 202